FShm: High-Rate Frame Manipulation in Kernel and User-Space

The high performance, low cost, and flexibility of commodity hardware systems make them appealing for network processing applications. However, the standard software architecture of such systems imposes significant limitations. At high rates (e.g., Gigabit Ethernet) and small frame sizes (64 byte) each frame must be processed in less than 672 ns. System calls, synchronization, and memory latencies can dominate this processing time. Despite significant effort to remove this overhead, we are aware of no general purpose mechanism that can handle this load on commodity hardware. This paper describes the frame-shared-memory architecture (FShm), a general-purpose software architecture for processing network frames on commodity multiprocessor hardware. FShm supports kerneland userspace processing at Gigabit Ethernet rates by increasing throughput, without reducing the per-frame processing time, by pipelining work across multiple processors. FShm can generate, capture, and forward frames at the theoretical maximum rate on a Gigabit Ethernet network for all frame sizes greater than 96 bytes, and at 95% of maximum for the 64 byte minimum frame size (the limit of the tested hardware).